<!DOCTYPE html>
<html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " lang="en">
<head>
<meta charset="utf-8">
<meta name="description" content="Tj2's Blog">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>绿萝间 (old posts, page 197) | 绿萝间</title>
<link href="assets/css/all-nocdn.css" rel="stylesheet" type="text/css">
<link href="assets/css/ipython.min.css" rel="stylesheet" type="text/css">
<link href="assets/css/nikola_ipython.css" rel="stylesheet" type="text/css">
<meta name="theme-color" content="#5670d4">
<meta name="generator" content="Nikola (getnikola.com)">
<link rel="alternate" type="application/rss+xml" title="RSS" href="rss.xml">
<link rel="canonical" href="https://muxuezi.github.io/index-197.html">
<link rel="prev" href="index-198.html" type="text/html">
<link rel="next" href="index-196.html" type="text/html">
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
    tex2jax: {
        inlineMath: [ ['$','$'], ["\\(","\\)"] ],
        displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
        processEscapes: true
    },
    displayAlign: 'center', // Change this to 'center' to center equations.
    "HTML-CSS": {
        styles: {'.MathJax_Display': {"margin": 0}}
    }
});
</script><!--[if lt IE 9]><script src="assets/js/html5.js"></script><![endif]-->
</head>
<body>
<a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>

<!-- Menubar -->

<nav class="navbar navbar-inverse navbar-static-top"><div class="container">
<!-- This keeps the margins nice -->
        <div class="navbar-header">
            <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-navbar" aria-controls="bs-navbar" aria-expanded="false">
            <span class="sr-only">Toggle navigation</span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            </button>
            <a class="navbar-brand" href="https://muxuezi.github.io/">

                <span id="blog-title">绿萝间</span>
            </a>
        </div>
<!-- /.navbar-header -->
        <div class="collapse navbar-collapse" id="bs-navbar" aria-expanded="false">
            <ul class="nav navbar-nav">
<li>
<a href="archive.html">Archive</a>
                </li>
<li>
<a href="categories/">Tags</a>
                </li>
<li>
<a href="rss.xml">RSS feed</a>

                
            </li>
</ul>
<ul class="nav navbar-nav navbar-right"></ul>
</div>
<!-- /.navbar-collapse -->
    </div>
<!-- /.container -->
</nav><!-- End of Menubar --><div class="container" id="content" role="main">
    <div class="body-content">
        <!--Body content-->
        <div class="row">
            
            

    
<div class="postindex">
    <article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/dlott-15147-2015-12-16-report.html" class="u-url">大乐透15147期(2015-12-16)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/dlott-15147-2015-12-16-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-17T08:00:00+08:00" title="2015-12-17 08:00">2015-12-17 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/dlott-15147-2015-12-16-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/slott-2015147-2015-12-15-report.html" class="u-url">双色球2015147期(2015-12-15)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/slott-2015147-2015-12-15-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-16T08:00:00+08:00" title="2015-12-16 08:00">2015-12-16 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/slott-2015147-2015-12-15-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/dlott-15146-2015-12-14-report.html" class="u-url">大乐透15146期(2015-12-14)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/dlott-15146-2015-12-14-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-15T08:00:00+08:00" title="2015-12-15 08:00">2015-12-15 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/dlott-15146-2015-12-14-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/slott-2015146-2015-12-13-report.html" class="u-url">双色球2015146期(2015-12-13)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/slott-2015146-2015-12-13-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-14T08:00:00+08:00" title="2015-12-14 08:00">2015-12-14 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/slott-2015146-2015-12-13-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/sqlalchemy-introduce.html" class="u-url">SQLAlchemy Introduce</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/sqlalchemy-introduce.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-13T19:17:02+08:00" title="2015-12-13 19:17">2015-12-13 19:17</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div tabindex="-1" id="notebook" class="border-box-sizing">
    <div class="container" id="notebook-container">

<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="perface">perface<a class="anchor-link" href="posts/sqlalchemy-introduce.html#perface">¶</a>
</h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>我们每天都要面对数据，数据库CRUD操作的能力对每个任务都至关重要。无论你是开发网页，桌面或其他应用，他们都需要快速且安全的接入数据。关系型数据库仍然是储存数据最主流的手段之一。</p>
<p>SQL是操作数据的利器，不过有时候要整合到应用里非常麻烦。作为一个Pythoner，你可能需要通过ODBC接口或者数据库API，用一些字符串生成查询。虽然这些方法可以有效的处理数据，但是数据安全和修改变得很困难。</p>
<p>这本书将介绍一个非常强大且具有灵活性的Python库SQLAlchemy，可以跨越关系型数据库与传统编程直接的鸿沟。SQLAlchemy允许你使用原始的SQL语言查询，更鼓励使用更高级的Pythonic展示方式和更友好的方法实现数据的查询和更新。它提供工具让你一次性把应用的类和对象映射成数据库的表，然后就可以不再理会表结构了，还可以让你不断优化模型的性能。</p>
<p>SQLAlchemy功能强大且充满柔性，但它还是有点复杂。SQLAlchemy教程只介绍了它的冰山一角，尽管在线文档也很全，但是更适合做参考书，不适合做教程。这本书的目的是要成为一个学习工具和一个方便的参考，帮你快速完成任务。</p>
<p>本书内容用的SQLAlchemy是1.0版本；但是之前的版本都可以用。0.8版可能需要一点小改动，大部分都是源自0.5版。</p>
<p>这本书分成三部分：SQLAlchemy Core，SQLAlchemy ORM和一个Cookbook部分。前两部分尽可能的做到一致性。我们有意在两个部分用同意的例子，目的是让你更方便的比较两种方式的用法。因此，你也可以根据自己的需要阅读本书SQLAlchemy Core和SQLAlchemy ORM的章节。</p>
<p class="more"><a href="posts/sqlalchemy-introduce.html">Read more…</a></p>
</div>
</div>
</div>
</div>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/postgres-import-csv-cp936-error.html" class="u-url">Postgres Import CSV cp936 Error</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/postgres-import-csv-cp936-error.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-13T19:11:05+08:00" title="2015-12-13 19:11">2015-12-13 19:11</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div tabindex="-1" id="notebook" class="border-box-sizing">
    <div class="container" id="notebook-container">

<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Postgres导入CSV中文编码错误">Postgres导入CSV中文编码错误<a class="anchor-link" href="posts/postgres-import-csv-cp936-error.html#Postgres%E5%AF%BC%E5%85%A5CSV%E4%B8%AD%E6%96%87%E7%BC%96%E7%A0%81%E9%94%99%E8%AF%AF">¶</a>
</h2>
<p>现在公司数据资源非常丰富，动辄要处理的csv就1E8行纪录，10G，分析起来相当爽。编码格式是cp936，用官方的COPY命令导入Postgres，一直出错，目前COPY的ENCODING不支持cp936。试着用Pandas读几行，保持csv为utf-8再COPY没问题。</p>
<p>解决办法：通过Python3的Pandas读取CSV转换成utf－8，再利用sqlalchemy＋psycopg2（目前只支持到Python3.4，还不支持3.5）导入即可。</p>
<blockquote>
<p>这么多依赖推荐使用<a href="https://www.continuum.io/downloads">Anaconda</a>，十分方便，尤其适合解决在Windows平台编译C语言相关库（lxml，psycopg2，numpy，pandas）时的各种奇葩问题。</p>
</blockquote>
<p>具体方法如下：</p>
<p class="more"><a href="posts/postgres-import-csv-cp936-error.html">Read more…</a></p>
</div>
</div>
</div>
</div>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/dlott-15145-2015-12-12-report.html" class="u-url">大乐透15145期(2015-12-12)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/dlott-15145-2015-12-12-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-13T08:00:00+08:00" title="2015-12-13 08:00">2015-12-13 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/dlott-15145-2015-12-12-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/slott-2015145-2015-12-10-report.html" class="u-url">双色球2015145期(2015-12-10)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/slott-2015145-2015-12-10-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-11T08:00:00+08:00" title="2015-12-11 08:00">2015-12-11 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/slott-2015145-2015-12-10-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/dlott-15144-2015-12-09-report.html" class="u-url">大乐透15144期(2015-12-09)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/dlott-15144-2015-12-09-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-10T08:00:00+08:00" title="2015-12-10 08:00">2015-12-10 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/dlott-15144-2015-12-09-report.html">Read more…</a></p>
</div>
    </div>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title"><a href="posts/slott-2015144-2015-12-08-report.html" class="u-url">双色球2015144期(2015-12-08)数据分析报告</a></h1>
        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                Tao Junjie
            </span></p>
            <p class="dateline"><a href="posts/slott-2015144-2015-12-08-report.html" rel="bookmark"><time class="published dt-published" datetime="2015-12-09T08:00:00+08:00" title="2015-12-09 08:00">2015-12-09 08:00</time></a></p>
        </div>
    </header><div class="p-summary entry-summary">
    <div>
<p>如有雷同，纯属巧合</p>
<p class="more"><a href="posts/slott-2015144-2015-12-08-report.html">Read more…</a></p>
</div>
    </div>
    </article>
</div>

        <nav class="postindexpager"><ul class="pager">
<li class="previous">
                <a href="index-198.html" rel="prev">Newer posts</a>
            </li>
            <li class="next">
                <a href="index-196.html" rel="next">Older posts</a>
            </li>
        </ul></nav><script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"> </script><script type="text/x-mathjax-config">
MathJax.Hub.Config({
    tex2jax: {
        inlineMath: [ ['$','$'], ["\\(","\\)"] ],
        displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
        processEscapes: true
    },
    displayAlign: 'center', // Change this to 'center' to center equations.
    "HTML-CSS": {
        styles: {'.MathJax_Display': {"margin": 0}}
    }
});
</script>
</div>
        <!--End of body content-->

        <footer id="footer">
            Contents © 2017         <a href="mailto:muxuezi@gmail.com">Tao Junjie</a> - Powered by         <a href="https://getnikola.com" rel="nofollow">Nikola</a>         
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0">
<img alt="Creative Commons License BY-NC-SA" style="border-width:0; margin-bottom:12px;" src="http://i.creativecommons.org/l/by-nc-sa/4.0/80x15.png"></a>
            
        </footer>
</div>
</div>


            <script src="assets/js/all-nocdn.js"></script><script>$('a.image-reference:not(.islink) img:not(.islink)').parent().colorbox({rel:"gal",maxWidth:"100%",maxHeight:"100%",scalePhotos:true});</script><!-- fancy dates --><script>
    moment.locale("en");
    fancydates(0, "YYYY-MM-DD HH:mm");
    </script><!-- end fancy dates --><script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-51330059-1', 'auto');
  ga('send', 'pageview');

</script>
</body>
</html>
